Posterior Predictions
(s/o to Andrew Heiss and his wonderful blog posts on these topics, highly recommend 1, 2)
Working with Posterior Distributions
\[
\mathbb{E}\left (y | \mathbf{X}\right ) = \mu = g^{-1}\left (\mathbf{X}\beta \right ) \\
p\left (y|x\right ) \sim \pi\left (\mu,...\right )
\]
Linear Predictions: \(\mathbf{X}\beta\)
Expected Value Predictions: \(\mathbb{E}\left (y | \mathbf{X}\right ) = g^{-1}\left (\mathbf{X}\beta \right )\)
Posterior Predictions: \(p\left (\tilde{y}|y \right)\)
Linear Predictions
\[
\mathbb{E}\left (y | \mathbf{X}\right ) = \mu = g^{-1}\left (\mathbf{X}\beta \right ) \\
p\left (y|x\right ) \sim \pi\left (\mu,...\right )
\]
Linear Predictions are draws from the linear prediction \(\mathbf{X}\beta\) that are linear combinations of the columns of our design matrix before transforming them with the link function.
In logistic regression, this would be predictions on the log odds scale (“what are the predicted log odds of being registered to vote?”)
Expected Value Predictions
\[
\mathbb{E}\left (y | \mathbf{X}\right ) = \mu = g^{-1}\left (\mathbf{X}\beta \right ) \\
p\left (y|x\right ) \sim \pi\left (\mu,...\right )
\]
Expected Value Predictions are draws of the expected values \(\mu\) that are linear predictions that have been transformed using the link function.
In logistic regression, this would be predicted probabilities (“what is the predicted probability of being registered to vote?”)
Posterior Predictions
\[
\mathbb{E}\left (y | \mathbf{X}\right ) = \mu = g^{-1}\left (\mathbf{X}\beta \right ) \\
p\left (y|x\right ) \sim \pi\left (\mu,...\right )
\]
Posterior Predictions are draws of data \(y_{rep}\) that are essentially new samples of data that are based on the posterior distribution from the model \(p(\tilde{y} \mid y)\).
In logistic regression, this would be new samples of binary (0/1) data that are based on the parameters learned by the model.
Working With Posterior Distributions
Andrew Heiss made this great cheat sheet (though his notation is a little different than ours).